Outlier Detection in High Dimensional, Spatial and Sequential Data Sets

نویسندگان

  • Sanjay Chawla
  • Joseph Davis
  • Pei Sun
  • Bavani Arunasalam
چکیده

Of all the data mining techniques, outlier detection seems closest to the definition of “discovering nuggets of information” in large databases. When an outlier is detected, and determined to be genuine, it can provide insights, which can radically change our understanding of the underlying process. The purpose of the research underlying this thesis was to investigate and devise methods to mine for outliers in different type of data sets. In this thesis, we propose: • An efficient sampling based outlier detection method for large high-dimensional data. • A measure, Spatial Local Outlier Measure (SLOM), which captures the local behavior of datum in their spatial neighborhood. With the help of SLOM, we are able to discern local spatial outliers, which are usually missed by global techniques like “three standard deviations away from the mean”. • A unique approach to mine for sequential outliers using Probabilistic Suffix Trees (PST).

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Outlier Detection and Visualisation in High Dimensional Data

The outlier detection problem has important applications in the field of fraud detection, network robustness analysis, and intrusion detection. Such applications have to deal with high dimensional data sets with hundreds of dimensions. However, in high dimensional space, the data are sparse and the notion of proximity fails to retain its meaningfulness. Many recent algorithms use heuristics suc...

متن کامل

Robust high-dimensional semiparametric regression using optimized differencing method applied to the vitamin B2 production data

Background and purpose: By evolving science, knowledge, and technology, we deal with high-dimensional data in which the number of predictors may considerably exceed the sample size. The main problems with high-dimensional data are the estimation of the coefficients and interpretation. For high-dimension problems, classical methods are not reliable because of a large number of predictor variable...

متن کامل

A Novel Subspace Outlier Detection Approach in High Dimensional Data Sets

Many real applications are required to detect outliers in high dimensional data sets. The major difficulty of mining outliers lies on the fact that outliers are often embedded in subspaces. No efficient methods are available in general for subspace-based outlier detection. Most existing subspacebased outlier detection methods identify outliers by searching for abnormal sparse density units in s...

متن کامل

Application of Recursive Least Squares to Efficient Blunder Detection in Linear Models

In many geodetic applications a large number of observations are being measured to estimate the unknown parameters. The unbiasedness property of the estimated parameters is only ensured if there is no bias (e.g. systematic effect) or falsifying observations, which are also known as outliers. One of the most important steps towards obtaining a coherent analysis for the parameter estimation is th...

متن کامل

Spatio-Temporal Outlier Detection Technique

Outlier detection is very important functionality of data mining, it has enormous applications. This paper proposes a clustering based approach for outlier detection using spatio-temporal data. It uses three step approach to detect spatiotemporal outliers. In the first step of outlier detection, clustering is performed on the spatio-temporal dataset with proposed Spatio-Temporal Shared Nearest ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2006